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The academic skills of today’s teenagers are diminishing, and are a cause for concern. One of the 
academic areas in need of improvement is science. The purpose of this causal comparative study was to 
determine the effect that a science-enhanced, curriculum would have on students’ achievement in 
science. The population for this study consisted of students in secondary agricultural education 
programs whose instructors held a science credential in Oklahoma during the 2009—2010 school year 
and were selected by state staff in the Agricultural Education Division of the Oklahoma Career and 
Technology Education Department to teach the science-enhanced, curriculum. In addition, 10 equally 
credentialed instructors formed a purposeful comparison group and were selected according to specific 
variables (e.g., similarity of students ’ SES status) for equivalency purposes. The findings of this study 
revealed that a statistically significant difference in students ’ science achievement did not exist as a result 
of the treatment. However, small practical differences were detected between the groups, as student 
performance in the treatment group was more than two and one-hcilf points higher than the means of 
students’ performance scores in the comparison group. Recommendations point to the need for 
replication of the study over an entire school year. 
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Introduction and Conceptual Framework 

The academic skills of today’s teenagers are 
diminishing, and a cause for concern exists 
among both state and national officials 
(Cavanagh, 2004). One of the academic areas in 
need of improvement is science (Dickinson & 
Jackson, 2008; National Center for Education 
Statistics, 2005; Provasnik, Gonzales, & Miller, 
2009). The National Commission on Excellence 
in Education (NCEE) identified that a “. . . 
widespread public perception that something is 
seriously remiss in our educational system” 
(NCEE, 1983, p. 1) exists. In addition, the 


report, A Nation at Risk: The Imperative for 
Educational Reform, stated that, “The 
educational foundations of our society are being 
eroded by a rising tide of mediocrity” (NCEE, 
1983, p. 5). Lloyd (1992) posited that as a result 
of these educational reports, evidence exists to 
support the need for educational change. 

Reports on the success of students from 
across the globe in comparison to the 
achievements of those in the United States 
indicate that American students are falling 
behind in science achievement when compared 
to other countries (National Center for 
Education Statistics, 2005; Provasnik et al., 


15 



Haynes, Robinson, Edwards, & Key 


Assessing the Effect... 


2009). Further, it appears as though progress in 
science achievement of American students has 
stagnated. As of 2007, the United States was 
ranked tenth out of 47 countries that participated 
in the Trends in International Mathematics and 
Science Study (TIMSS). Countries out-ranking 
American students in science achievement 
scores were Singapore, Chinese Taipei, Japan, 
Korea, England, Hungary, the Czech Republic, 
Slovenia, and the Russian Federation (National 
Center for Education Statistics, 2005). 

Cavanagh (2004) noted that, according to 
the American College Testing (ACT) program, 
78% of students who took a college entrance 
examination were deficient in the areas of 
mathematics, science, and English. Thus, it was 
determined that these students were ill-prepared 
for college-level coursework, justifying the need 
for improvements at the secondary level. 
Further, it was noted in the latest Program for 
International Student Assessment (PISA) that, 
“U.S. 15-year-olds are not able to apply 
scientific knowledge and skills to real world 
tasks as well as their peers . . .” (Provasnik et al., 
2009, p. 45). 

Provasnik et al. (2009) compared the 
average science scale scores of students in the 
United States to international students in the 
areas of reading, mathematics, and science. It 
was determined that Oklahoma ranked 28th in 
the nation out of the 45 states that reported 
science achievement scores. This figure is 
discouraging and serves as an indicator of the 
lack of preparedness of Oklahoma students for 
higher education and the real world. 

Secondary agricultural education exists to 
prepare people for college and careers (Roberts 
& Ball, 2009). Because it has long been lauded 
as the world’s oldest science (Ricketts, Duncan, 
& Peake, 2006), agricultural education strives to 
help students understand scientific principles 
and concepts better in the context of agriculture 
(Thompson & Balschweid, 2000). As such, 
agricultural education could serve as an effective 
medium to convey scientific terminology, 
principles, and those concepts that are inherent 
to botany and zoology (Parr & Edwards, 2004). 

Because agricultural education holds the 
potential to aid students effectively in 
understanding science better through the context 
of agriculture, curricula developed toward this 
end should be made available in the secondary 
classroom. One such curriculum is available 


through the Center for Agricultural and 
Environmental Research and Training 
(CAERT). CAERT provides agriculturally- 
based, science-enhanced materials for use in 
agricultural and environmental instructional 
areas at the secondary level (CAERT, 2010a). 
Specializing in activities that are collaborative 
by nature, students of agricultural education are 
provided a curriculum that is intended to allow 
them to be more involved and engaged in the 
learning process (CAERT, 2010a). 

Conceptually, this study was based on the 
premise of contextual teaching and learning 
(CTL). CTL “is a system of instruction based 
on the philosophy that students learn when they 
see meaning in academic material, and they see 
meaning in schoolwork when they can connect 
new information with prior knowledge and their 
own experience” (Johnson, 2002, p. vii). CTL 
involves the learning situation, the content being 
learned, and the opportunity to apply the content 
in other contexts (Clough & Lehr, 1996). 
Because the brain is a parallel processor and can 
perform functions and activities between 
contexts simultaneously (Caine & Caine, 1995), 
CTL is an effective method for helping students 
understand the meaning and relevance of how 
specific content applies to their daily lives 
(Berns & Erickson, 2001). Through the CTL 
method, students are able to transfer better their 
knowledge from one environment to another 
(Clough & Lehr, 1996; Edling, 1993). Because 
students use their prior knowledge to make 
meaning and solve problems when instructed 
through CTL (Berns & Erickson, 2001), the 
central question of this study was, “Does 
teaching science concepts in an agricultural 
context help students learn science principles 
better?” 

Statement of the Problem 

High stakes tests have placed increased 
requirements on schools to raise students’ test 
scores in science. Moreover, the ever- 
increasing demand for workers who are 
scientifically literate and capable of applying 
their understanding of science in the workplace 
continues to be an escalating imperative. 
Agricultural education at the secondary level, 
including animal science and horticulture 
curriculums, is based inherently on fundamental 
science principles and concepts. However, little 
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empirical evidence exists that demonstrates 
whether teaching a science-enhanced 
curriculum in the context of animal or plant 
science courses would affect student 
achievement in science positively. 

Purpose of the Study 

The purpose of this study was to determine 
if a science-enhanced curriculum (i.e., CAERT) 
taught in a secondary level animal science or 
horticulture course would improve students’ 
understanding of selected scientific principles 
significantly, when compared to students who 
were instructed using a traditional curriculum. 
The following objectives and hypothesis guided 
this study. 

1. Determine the personal characteristics (i.e., 
gender, age, grade classification, Biology 1 
End of Instruction score, race/ethnicity, and 
number of agricultural education courses 
taken) of students enrolled in selected 
animal science or horticulture courses in 
Oklahoma during spring semester 2010. 

2. Determine the effect of a science-enhanced 
(i.e., CAERT) curriculum on students’ 
science achievement, as determined by a 
science proficiency examination. 

H 0 1: The science achievement of students who 
received the science-enhanced CAERT 
curriculum in animal science or horticulture 
will not differ significantly (i.e., p > .05) 
from those students who were taught the 
traditional animal science or horticulture 
curriculum, as measured by the TerraNova 3 
science achievement examination (H 0 1: 

P 1 treatment group = P 2comparison group) • 

Methodology 

The population for this study consisted of 
students whose secondary agricultural education 
instructors held a science credential in 
Oklahoma during the 2009-2010 school year. 
The puiposeful sample consisted of 10 treatment 
groups, including students whose teachers were 
selected by Agricultural Education Division staff 
of the ODCTE to use the science-enhanced 
CAERT curriculum developed for the 
instruction of animal science and horticulture 
courses during the 2009-2010 school year. In 


addition, students of 10 different instructors 
formed a purposeful comparison group. These 
teachers also held a science credential and were 
selected according to specific school and student 
data obtained from the 2008-2009 
Computerized Enrollment System for Instructors 
(CESI) report. The CESI report is used by the 
ODCTE’s, Information Management Division to 
collect selected information of Oklahoma 
secondary agricultural education programs and 
their students. Therefore, schools that 
“matched” the treatment group, based on review 
of established criteria, were selected to provide 
an appropriate counterfactual group (Creswell, 

2008) for the comparison of students’ test 
scores. 

The criteria used in this study were 
established and recommended by the National 
Research Center for Career and Technical 
Education (NRCCTE) (J. Stone, personal 
communication, December 3, 2009), who also 
provided funding for the use and scoring of the 
TerraNova 3 science achievement examinations. 
The criteria considered for selection of the 
counterfactual group included agricultural 
education instructors who held a teaching 
certification in science at the time of the study, 
as well as academic performance index (API) 
scores of their schools, and socioeconomic status 
(SES) of the student participants. Random 
sampling was used to select students to take the 
science examination. The instructors’ 
classrooms served as the study’s units of 
analysis for purposes of comparison. 

The design of the study was ex post facto, 
causal comparative because no random 
assignment of the treatment group occurred 
(Ary, Jacobs, & Razavieh, 2002). The treatment 
group was pre-determined through selection of 
instructors by ODCTE staff, i.e., agricultural 
education teachers who received access to the 
CAERT curriculum. The curriculum was 
designed to explicate and reinforce scientific 
principles through the instruction of select 
agricultural education courses, including 
modules supported by downloadable lesson 
plans, aligned learning standards, summary 
reports, PowerPoint® files, and E-Units (K. 
Murray, personal communication, October 1, 

2009) . E-Units are online student text resources 
that are designed to reinforce the lesson plans 
that were a part of the CAERT science- 
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enhanced, curriculum (D. Pentony, personal 
communication, December 6, 2010). 

The CAERT curriculum was selected for use 
because it was developed according to standards 
for agricultural education in Oklahoma, was 
acceptable as science credit for college entrance 
puiposes, and consisted of an online delivery 
method. As a result of the state alignment, the 
animal science curriculum included 28 units 
with 160 instructional lessons, and the 
horticulture curriculum included 29 units with 
148 lessons (CAERT, 2010b). The unique 
purpose of CAERT is that it is a science- 
enhanced curriculum not otherwise offered by 
curriculum providers for use in Oklahoma (K. 
Murray, personal communication, October 1, 
2009). 

The treatment group teachers were provided 
access to the CAERT curriculum via passwords 
and user names in summer 2009. These teachers 
were instructed by ODCTE state staff members 
to become familiar with the modules pertaining 
to animal science and horticulture prior to the 
beginning of the up-coming fall semester. In 
addition, this group of teachers was brought onto 
the ODCTE campus for a one-half day training 
seminar during September 2009 to receive an 
overview of the curriculum (i.e., the functions of 
the curriculum and how to use its teaching 
resources). For the purpose of testing this 
study’s intervention (i.e., the CAERT 
curriculum), a purposeful comparison group was 
selected from the same list of agricultural 
education teachers who had achieved science 
certification in Oklahoma (N = 40). This group 
was instructed to teach their courses (i.e., animal 
science or horticulture) using the same 
curriculum as they always had in the past. In 
most cases, this was the curriculum produced by 
the Curriculum for Instructional Materials 
Center (CIMC), which is created specifically for 
agriculture teachers in Oklahoma. 

To determine equivalency of the treatment 
and comparison groups, student performance 
was compared on the Oklahoma Department of 
Education’s End of Instruction (EOI) 
examination in science. In addition, school 
district’s academic performance index and 
accountability data (API), and the schools’ 
percentage of low income clientele served by the 
free and reduced lunch program (SES) were 
compared. 


The Oklahoma Department of Education’s 
EOI examination in science is a part of a larger, 
statewide testing program known as the 
Oklahoma School Testing Program (OSTP) 
(Oklahoma State Department of Education, 
2010a). Students completing an area of 
instruction are expected to pass the 
corresponding standardized assessment. EOI 
examinations are designed to assess a student’s 
level of competency relative to the Priority 
Academic Student Skills (PASS), which are 
Oklahoma-based content standards (Oklahoma 
State Department of Education, 2010b). 

Evaluation of student competency in 
Biology involved the use of core curriculum test 
scores for Biology in Oklahoma. These core 
curriculum tests in Oklahoma are administered 
in accordance with students’ ability level as 
established by local school administration and 
admission, review, and dismissal (ARD) 
meetings. The two types of core curriculum 
tests used to measure students’ science 
achievement include the Biology I End of 
Instruction test, which is administered to the 
general school population, and the Oklahoma 
Modified Alternate Assessment Program 
(OMAAP) test, which is administered to those 
students qualifying for modified testing as a 
result of local ARD meetings. 

Four performance levels exist to classify 
student achievement. For the regular test 
administration (i.e., EOI), performance levels 
are divided into advanced (755 - 999), 
satisfactory (691 - 774), limited knowledge (627 
- 690), and unsatisfactory (440 - 626). The 
alternate test administration (OMAAP) is 
divided into four performance levels. They 
consist of advanced (265 - 350), satisfactory 
(250 - 264), limited knowledge (233 - 249), and 
unsatisfactory (100 - 232) (Oklahoma State 
Department of Education, 2010c). EOI 

categorical scores were coded as 1 = 

unsatisfactory, 2 = limited knowledge, 3 = 
satisfactory, and 4 = advanced for comparison 
purposes between the regular and alternate test 
administrations (Oklahoma State Department of 
Education, 2010a). 

The Academic Performance Index (API) for 
Oklahoma was developed based on the need to 
compare school performance to meet 
requirements established by Oklahoma law, as 
well as legislation pursuant to Public Law 107- 
110, commonly referred to as No Child Left 
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Behind (Oklahoma State Department of 
Education, 2010c). API scores range from 0 to 
1500, with the most recent reported state 
average being 1279 (Oklahoma State 
Department of Education, 2010c). Components 
of a school’s API include EOI scores, Academic 
Excellence as measured by students’ 
participation in the ACT college entrance 
examination, remediation rates for college 
students in reading and mathematics, and school 
completion, as determined by student attendance 
coupled with graduation and dropout rates 
(Oklahoma State Department of Education, 
2010c). To ensure equivalency of the treatment 
and comparison groups, schools were compared 
on the basis of EOI scores, API, and 
socioeconomic status (SES). 

When comparing these variables for 
equivalency, the treatment group had an EOI 
group mean score of 2.67 (2 = limited 
knowledge) ( SD = 1.12). The mean score for 
the comparison group was 2.88 (2 = limited 
knowledge) ( SD = .93). The treatment group 


had an API group mean score of 1387.00 (SD = 
57.42); the mean score for the comparison group 
was 1295.86 (SD = 74.40). The treatment group 
had a SES group mean score of 44.85 (SD = 
13.94). The comparison group had a mean score 
of 43.53 (SD = 9.40) for SES. An independent 
samples t-test comparison of the treatment and 
comparison groups did not reveal a statistically 
significant difference in student science 
knowledge (p = .580) prior to the treatment at an 
alpha level of .05 (see Table 1). 

Independent samples t-tests were used to 
compare the treatment and comparison group 
participants on the EOI, API, and SES variables. 
It was revealed that a statistically significant 
difference (see Table 1) in API scores existed 
between the two groups (p = .045) at an alpha 
level of .05. No significant differences were 
found for the other measure. However, the 
reader is cautioned on making generalizations 
beyond the sample examined in the study (see 
Table 1). 


Table 1 


Treatment and Comparison Group Equivalency According to A Comparison of EOI. API, and Socio- 
Economic Status Scores 


Groups 

Minimum & Maximum 

M 

SD 

t-value 

p-value 

EOI a 

1-4 

2.67 

1.12 

-.561 

.579 

EOI b 


2.88 

.93 



API 3 

0-1500 

1387.00 

57.42 

2.290 

.045’ 

API b 


1295.86 

74.40 



SES a 

0 - 100% 

44.85 

13.94 

.197 

.848 

SES b 

rr ; * : rr? 

43.53 

9.40 




a = Treatment; b = Comparison; p < .05 


To determine the effect that the CAERT 
curriculum had on students’ science 
achievement, a science proficiency examination 
was used. The TerraNova 3 Form G assessment 
series examination, designed and developed by 
CTB/McGraw-Hill (a subsidiary of The 
McGraw-Hill Companies, Inc.), was the 
examination used in this study. The 
examination consisted of normed sections that 
are designed to test student competencies in 
reading, language, mathematics, social studies, 
and science (CTB/McGraw-Hill LLC, 2008). 
“A normed section is a subset of TerraNova 
Third Edition for which scores from a nationally 
representative norm group are available” 
(CTB/McGraw-Hill LLC, 2008, p. 1). The 
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normed section for science consisted of 40 
multiple choice questions designed to assess 
student competence in science. The students 
were provided four answers for each multiple 
choice question from which to determine the 
correct answer. School district testing liaisons 
arranged for and proctored the science 
examination. 

The NRCCTE agreed to provide science 
examinations and their scoring for 80 students in 
the study (i.e., four to five students per 
classroom, treatment and counterfactual). An 
online calculator was used to estimate the 
appropriate sample size needed for this study 
(Soper, 2010). It was determined that if three 
covariates were used for prediction, 76 
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participants were needed to accommodate an 
alpha level of .05, with an anticipated effect size 
of .15, and a desired power level of .80. For 
practical testing purposes, 80 treatment and 
comparison students were chosen randomly 
from the 20 classrooms in the study to 
participate in taking the science examination. 
This allowed the researcher to select four to five 
students randomly per classroom to achieve the 
appropriate sample size for the study. In all, 80 
students were randomly selected to ensure a 
strong power analysis and effect size for the 
study. Power is determined typically by sample 
size (Keppel, 1991) and is defined as, “the 
probability of correctly rejecting a false null 
hypothesis” (Shavelson, 1996, p. 314). 
Therefore, one means to increase power is to 
increase sample size. As power increases, so 
does the magnitude of the effect, or effect size 
(Shavelson, 1996). “Effect size is the 
discrepancy between the null hypothesis and the 
alternative hypothesis of interest” (Shavelson, 
1996, p. 317). 

Statistical analysis for the study was 
completed with the Predictive Analytics 
Software (PASW) 18.0 and Microsoft Excel 
2007. To assess research question one, students 
were asked to identify characteristics pertaining 
to their gender, age, grade classification, and 
race/ethnicity. To summarize trends and 
tendencies relating to the personal characteristics 
data, descriptive statistics (i.e., mean, median, 
mode, frequency, and percentages) were 
computed. To assess research question two, an 
independent samples t-test was used. Ary, 
Jacobs, and Razavieh (2002) identified that a t— 
test for independent samples serves as an ideal 
statistical procedure for determining statistically 
significant differences between groups. 


Effect sizes were also calculated to 
determine what practical effect the treatment had 
on the post-treatment measures of the study 
(i.e., animal science and horticulture agricultural 
competency examinations). The effect size was 
calculated per Cohen’s (1988) procedure. 
According to Cohen (1988), effect size is 
calculated and compared to three benchmark 
standards: small effect size (d = .20), medium 
effect size (d = .50), and large effect size (d = 
.80). However, research by Thompson (2002) 
indicated that adherence to this standard may be 
too stringent and that the effect itself is 
determined by what has been studied. For 
example, large effect sizes can be considered 
trivial when applied to outcomes that are trivial 
(Trusty, Thompson, & Petrocelli, 2004). So, the 
benchmark standards to interpret effect size for 
selected results of this study (as calculated by 
Cohen’s [1988] formula) were expanded and 
compared to the following standard proffered by 
Thalheimer and Cook (2002) (see Table 2). 

Using Thalheimer and Cook (2002), the 
relative size of a negligible effect must be 
greater than or equal to - 0.15 and less than .15. 
To be classified as having had a small effect, the 
relative size must be greater than or equal to .15 
but below .40. A medium effect classification 
must be greater than or equal to .40 but less than 
.75 in relative size. Those effect sizes that are 
considered to be large must have a relative size 
of greater than or equal to .75 but less than 1.10. 
To have an effect size classified as very large, 
the relative size must be greater than or equal to 
1.10 but less than 1.45. Finally, to have an 
effect size considered huge, the relative size 
must be greater than 1.45. 


Table 2 


Relative Size of Cohen’s d According to Thalheimer and Cook (2002) 


Effect Size Classification 

Relative Size 

Negligible Effect 

> = - 0.15 and < .15 

Small Effect 

> = .15 and <.40 

Medium Effect 

> = .40 and < .75 

Large Effect 

> = .75 and < 1.10 

Very Large Effect 

>= 1.10 and < 1.45 

Huge Effect 

> 1.45 
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Agricultural teachers and their students from 
20 secondary agricultural education programs in 
the state of Oklahoma served as the subjects for 
this study and provided the data described in the 
findings section. However, mortality occurred 
during the study and affected the final sample 
size. Mortality is “a potential threat to internal 
validity in an experiment when individuals drop 
out during the experiment for any number of 
reasons” (Creswell, 2008, p. 642). 

Findings/Results 

Research question one sought to determine 
the personal characteristics (i.e., gender, age, 


grade classification, end of instruction score 
(EOI), number of agricultural science courses 
taken, and race/ethnicity) of students enrolled in 
the targeted courses involved in the study and 
who were selected randomly for testing (N = 
80). The students were asked for their personal 
characteristics information in conjunction with 
their post-test administrations. A total of 69 
students completed the questionnaire (treatment 
n = 29; comparison n = 40) administered during 
the post treatment testing process. The 
treatment group students included 13 males 
(45%) and 16 females (55%) (see Table 3). 


Table 3 


Selected Personal Characteristics of Treatment (n= 29) and Comparison (n= 40) Group Students 


Variable 

Treatment 

f % 

Comparison 

f % 

Gender 

Male 

13 

44.8 

18 

45.0 

Female 

16 

55.2 

22 

55.0 

Age 

14 

0 

0.0 

1 

2.5 

15 

1 

3.4 

5 

12.5 

16 

9 

31.0 

15 

37.5 

17 

6 

20.7 

11 

27.5 

18 years or older 

13 

44.8 

8 

20.0 

Race/Ethnicity 

White/Caucasian 

24 

82.8 

34 

85.0 

American Indian/Alaskan Native/Pacific Islander 

4 

13.8 

5 

12.5 

Other 

1 

3.4 

1 

2.5 

Grade Classification 

9th 

0 

0.0 

6 

15.0 

10th 

11 

37.9 

17 

42.5 

11th 

4 

13.8 

7 

17.5 

12 th 

14 

48.2 

10 

25.0 


None of the students in the treatment group 
were 14 years of age. One respondent was 15 
(3%), nine respondents were 16 (31%), six 
(21%) respondents were 17, and 13 (45%) 
respondents were 18 years of age or older. 

Regarding race/ethnicity of those who 
responded, 24 respondents (83%) self-selected 
their classification as White/Caucasian. None of 
the students reported they were African- 
American or Asian. Four (14%) students 
reported they were American Indian/Alaskan 
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Native/Pacific Islander, and one (3%) student 
selected his/her ethnicity as other (see Table 3). 

No respondents from the treatment group 
represented the ninth grade. Eleven of the 
respondents (38%) were tenth graders, four of 
the respondents (14%) were eleventh graders, 
and the other 14 students (48%) were twelfth 
graders (see Table 3). 

The comparison group students consisted of 
18 (45%) males and 22 (55%) females (see 
Table 3). One of the respondents (3%) was 14 
years of age, and five (13%) were 15 years of 
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age. Fifteen (38%) respondents were 16 years of 
age, 11(28%) were 17 years of age, and eight 
(20%) were 18 years of age or older (see Table 
3). 

As for race/ethnicity, 34 (85%) students 
classified themselves as White/Caucasian, five 
(13%) identified their race/ethnicity as being 
American Indian/ Alaskan Native/Pacific 
Islander, and one respondent (3%) selected the 
other classification. None of the students 
identified African-American or Asian as their 
race/ethnicity (see Table 3). 

None of the comparison group students were 
eighth graders. Rather, the students were 
distributed evenly across the remaining grade 
classification levels: six respondents (15%) were 
ninth graders, 17 (43%) were tenth graders, 
seven (18%) were eleventh graders, and 10 
(25%) were twelfth graders (see Table 3). 

Research question number two sought to 
determine the effect that a science-enhanced 
CAERT curriculum had on students’ science 
achievement, as determined by the TerraNova 3 
science proficiency examination. The science 
portion of the examination was administered 
after the treatment (i.e., teaching of the CAERT 


science-enhanced curriculum) to assess and 
compare the science achievement of the 
treatment and comparison group students. Data 
were analyzed and converted to percentages (0 - 
100) from raw data (0 - 40) for purposes of 
analysis using the following formula: 
Science—Enhanced Examination Raw 

Score/Total Raw Score X 100 — % Score 

The treatment group students (n = 29) who 
took the science-enhanced examination had a 
group mean score of 55.86 with a standard 
deviation of 16.55 (see Table 4). The 
comparison group students (n = 40) had a group 
mean score of 53.31 with a standard deviation of 
16.01. An independent samples f-test 
comparison of the treatment and comparison 
groups did not reveal a statistically significant 
difference in science achievement as a result of 
the treatment (t = .64; p = .522) at an alpha level 
of .05. Further, the effect size, calculated 
according to Thalheimer and Cook (2002), was 
small (d = .16) (see Table 4). 

As such, the null hypothesis (H 0 1) was 
accepted, indicating that the CAERT curriculum 
did not have a statistically significant effect (p < 
.05) on students’ science achievement. 


Table 4 


Science—Achievement Examination Scores of Treatment and Comparison Groups 


TerraNova 3 Examination 

Min. & Max. 

/ 

M 

SD 

f-value 

p-value 

Treatment 

0-100 

29 

55.86 

16.55 

.644 

.522“ 

Comparison 


40 

53.31 

16.01 




p < .05; “Effect size = Small (.16 per Cohen’s d\ Thalheimer & Cook, 2002) 


Limitations 

Certain conditions and variables important 
to this study were outside of the control of the 
researcher. For example, treatment teachers 
were selected purposefully by ODCTE state 
staff, which affected generalizability of the study 
due to non-randomization of the treatment 
teachers. Further, multiple attempts were made 
at collecting EOI data for those students who 
participated in the study. In Oklahoma, each 
school district “houses” its own student database 
(i.e., EOI results). As such, some schools were 
reluctant to release those data for the purpose of 
the study. Additionally, comparison of the 
schools by API scores found that the treatment 
group schools were statistically significantly 
different (i.e., at p < .05) than the comparison 


group schools, indicating a higher degree of 
aptitude overall. Finally, no incentives were 
provided for the teachers. Unfortunately, some 
teachers chose not to use the curriculum in its 
entirety or test their students accordingly. 

Conclusions 

This study found that a majority of those 
students who participated were female; fifty- 
five percent of the students in both groups were 
female. Further, in terms of race/ethnicity, the 
category representing the majority of both 
groups (treatment and comparison) was 
White/Caucasian. Finally, most students were 
16 years of age or older and belonged to the 
sophomore and senior classes primarily. 
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This study found that the use of the science- 
enhanced, CAERT curriculum did not result in a 
statistically significant increase (p < .05) in 
student performance as determined by the 
TerraNova 3 science proficiency examination. 
Therefore, the null hypothesis was not rejected. 
However, small practical differences were 
detected between the groups, as the student 
performance score mean of the treatment group 
was more than two and one-half points higher 
than the mean of students’ performance scores 
of the comparison group. Although not 
statistically significant, these results are similar 
to findings reported by Roegge and Russell 
(1990). The findings also suggest that students 
can learn academic content better when it is 
embedded in a familiar context (Caine & Caine, 
1995; Parr et al., 2009; Roberts & Ball, 2009; 
Young, Edwards, & Leising, 2009). 

Recommendations for Research 

Although the findings of this study did not 
indicate a statistically significant difference in 
science achievement for the treatment group 
students, the intervention (i.e., the science- 
enhanced, CAERT curriculum) may have 
potential in this area. However, additional 
research is needed. Because the treatment 
sample was pre-determined by ODCTE staff, 
the generalizability of this study suffered. So, 
this study should be replicated using a true 
experimental design in which teachers are 
selected randomly in an effort to generalize any 
future findings more broadly. A future 
investigation should occur with a different 
sample of teachers to determine if the science- 
enhanced CAERT curriculum was the 
determining factor in the outcome of the 
research that was conducted, or if it was a result 
of teacher effect. To answer this question, a 
hierarchical linear modeling (HLM) analysis 
could be conducted. 

This study lacked prolonged, sustained 
professional development regarding pedagogy 
needed to teach science content effectively (e.g., 
an inquiry-based teaching approach). However, 
the treatment group teachers were instructed to 
become familiar with the curriculum (i.e., the 
modules pertaining to animal science and 
horticulture) and were brought to the ODCTE 
campus for a one-half day training seminar 
during September 2009 for an overview of the 


functions of the curriculum and how to use its 
teaching resources. Therefore, from a 
pedagogical perspective (Brazen & Clark, 2005), 
future research should determine if a student- 
centered approach (e.g., inquiry-based teaching 
and learning) has an effect on students’ ability to 
learn science in the context of agriculture when 
compared to a teacher-centered approach. 

Recommendations for Practice 

The science achievement of students who 
received the study’s treatment yielded promising 
results. As a result of the findings of this study 
and others, (e.g.,Parr, Edwards, & Leising, 2006; 
Parr, Edwards, & Leising, 2009; Roegge & 
Russell, 1990; Young et al., 2009), 
improvements in student achievement can be 
realized as a result of teachers integrating 
curriculum. Put simply, students are capable of 
learning better when information is presented to 
them in a way that it relates to their personal 
experiences. Therefore, it is recommended that 
agriculture teachers collaborate with their 
science teacher colleagues in the development 
and reinforcement of learning resources that 
support and supplement the science aspects of 
the agriculture curriculum. 

Moreover, a community of practice should 
be established between agriculture teachers and 
their respective science teacher counterpart. 
Chalmers and Keown (2006) identified this as a 
cost-effective practice for providing 
professional development to teachers, which 
could also reinforce the self-efficacy of 
instructors in teaching the science content 
inherent to their curricula. Further, professional 
development should focus on helping instructors 
understand the use and format of the CAERT 
curriculum better. Specifically, workshops 
should focus on helping teachers learn ways to 
emphasize science concepts effectively as well 
as assist teachers in acquiring the pedagogical 
practices supporting inquiry-based teaching and 
learning. 

Discussion and Implications 

As a result of the curricular intervention, this 
study showed potential for improving student 
achievement in science when curriculum is 
taught in the context of agriculture. This 
implication is consistent with other studies that 


Journal of Agricultural Education 


23 


Volume 53, Number 2, 2012 



Haynes, Robinson, Edwards, & Key 


Assessing the Effect... 


emphasized science (e.g., Balschweid, 2002; 
Chiasson & Burnett, 2001; Ricketts et al., 2006; 
Roegge & Russell, 1990), as well as different 
academic areas such as math (e.g., Parr et al., 
2006; 2009; Young et al., 2009). Many of the 
instructors in this study had 21 or more years of 
teaching experience (Haynes, 2010) and all held 
a science endorsement or certification. Is it 
possible that having an additional teacher 
certification in science, some of the teachers 
may have actually taught science in Oklahoma 
before they became an agriculture teacher? If 
so, this could have been a confounding variable 
that affected the study’s results. 

Dewey (1938) argued for the integration of 
academics and vocational training as a way to 
reinforce the principles of learning thereby 
allowing for the development of life skills 
readily transferable across contextual areas. 
That position speaks to the potential for a 
science-enhanced curriculum being effective, 
regardless of students’ prior instructional 
experiences. 

Perhaps an increased amount of time 
exposing students to the science-enhanced, 
CAERT curriculum would have had a stronger 
effect on their science achievement. Parr et al. 


(2009), in their study on the selected effects of a 
curriculum integration intervention on the 
mathematics performance of secondary students 
enrolled in an agricultural power and technology 
course, stated that, “perhaps the short time 
period over which this study was conducted (i.e., 
one semester) did not allow sufficient 
opportunity for significant differences in student 
math achievement to emerge. . . ” (p. 66). The 
same statement could be applied to this study, 
perhaps. 

Likewise, maybe the short duration (i.e., the 
spring 2010 semester) during which this 
intervention occurred did not provide enough 
time for significant differences in students’ 
science achievement to appear. It is also 
possible that the comparison group teachers 
were doing a good job of emphasizing the 
science inherent to agriculture in their 
curriculum already. It is possible that instructors 
in Oklahoma were teaching a high level of 
science in their classes already. If so, this could 
account for the lack of a statistically significant 
difference in science achievement favoring the 
treatment group. Additional research should 
address these and related questions. 


References 


Ary, D., Jacobs, L. C., & Razavieh, A. (2002). Introduction to research in education (6th ed.). Belmont, 
CA: Wadsworth/Thomson Learning. 

Balschweid, M. A. (2002). Teaching biology using agriculture as the context: Perceptions of high school 
students. Journal of Agricultural Education, 43(2), 56-67. doi: 10.5032/jae.2002.02056 

Bems, R., & Erickson, P. M. (2001). Contextual teaching and learning: Preparing students for the new 
economy. The Highlight Zone: Research @ Work, 5. Retrieved from 
http://www.cord.org/uploadedfiles/NCCTE_Highlight05-ContextualTeachingLearning.pdf 

Brazen, E. F., & Clark, C. D. (2005). Promoting interactive learning with an electronic student response 
system. NACTA Journal, 49(3), 11-16. 

Caine, R. N., & Caine, G. (1995). Reinventing schools through brain-based learning. Educational 
Leadership, 52(7), 43-47. 

Cavanagh, S. (2004). Students ill-prepared for college, ACT warns. Education Week, 24(8), 5. 

Center for Agricultural and Environmental Research and Training. (2010a). Facts and mission. Retrieved 
from http://www.caert.net/mission.asp 

Center for Agricultural and Environmental Research and Training. (2010b). Product listings. Retrieved 
from http://www.caert.net/estorel/shopdisplayproducts.asp?id=17&cat=Multimedia 


Journal of Agricultural Education 


24 


Volume 53, Number 2, 2012 


Haynes, Robinson, Edwards, & Key 


Assessing the Effect... 


Chalmers, L., & Keown, P. (2006). Communities of practice and professional development. International 
Journal of Lifelong Education, 25(2), 139-156. doi: 10.1080/02601370500510793 

Chiasson, T. C., & Burnett, M. F. (2001). The influence of enrollment in agriscience courses on the science 
achievement of high school students. Journal of Agricultural Education, 42(1), 61-71. doi: 
10.5032/jae.2001.01061 

Clough, R. W., & Lehr, R. P. (1996). Testing knowledge of human gross anatomy in medical school: An 
applied contextual-learning theory method. Clinical Anatomy, 9, 263-268. doi: 
10.1002/(SICI)1098-2353(1996)9:4<263::AID-CA9>3.0.CO;2-F 

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence 
Erlbaum Associates. 

Creswell, J. W. (2008). Educational research: Planning, conducting, and evaluating cpiantitative and 
qualitative research (3rd ed.) Upper Saddle River, NJ: Pearson/Merrill Prentice Hall. 

CTB/McGraw-Hill LLC (2008). TerraNova 3 terranova norms book spring (3rd ed.). Monterey, CA: 
CTB/McGraw-Hill LLC. 

Dewey, J. (1938). Experience and education. New York, NY: Collier Books. 

Dickinson, G., & Jackson, J. K. (2008). Planning for success: How to design and implement project-based 
science activities. The Science Teacher, 75(8), 29-32. 

Edling, W. (1993). Contextual learning and tech prep curriculum integration. Available fromEBSCO host 
ERIC. Retrieved from 

http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=ED367787&site=ehost-live 

Haynes, J. C. (2010). Testing the effect of a science—enhanced curriculum on the science achievement and 
agricultural competency of secondary agricultural education students. (Unpublished doctoral 
dissertation). Oklahoma State University, Stillwater, OK. 

Johnson, E. B. (2002). Contextual teaching and learning: What it is and why it’s here to stay. Thousand 
Oaks, CA: Corwin Press, Inc. 

Keppel, G. (1991). Design and analysis: A researcher’s handbook (3rd ed.). Englewood Cliffs, NJ: Prentice 
Hall. 

Lloyd, B. W. (1992). A review of curricular changes in the general chemistry course during the twentieth 
century. Journal of Chemical Education, 69(8), 633-636. doi: 10.1021/ed069p633 

National Center for Education Statistics. (2005). NAEP state comparisons - data table. Retrieved from 

http://nces.ed.gov/nationsreportcard/statecomparisons/withinyear.aspx?usrSelections=l%2cSCI%2 

c0%2c2%2cwithin%2c0%2c0 

National Commission on Excellence in Education. (1983). A nation at risk: The imperative for educational 
reform. The Elementary School Journal, 84(2), 113-130. doi: 10.1086/461348 

Oklahoma State Department of Education. (2010a). Accountability and assessment. Retrieved from 
http://sde.state.ok.us/acctassess/core.html 


Journal of Agricultural Education 


25 


Volume 53, Number 2, 2012 


Haynes, Robinson, Edwards, & Key 


Assessing the Effect... 


Oklahoma State Department of Education. (2010b). Priority academic student skills (PASS). Retrieved from 
http://sde.state.ok.us/Curriculum/PASS/default.html 

Oklahoma State Department of Education. (2010c). Measuring success in Oklahoma schools. Retrieved 
from http://sde.state.ok.us /AcctAssess/pdf/APPBrochure.pdf 

Parr, B. A., Edwards, M. C., & Leising, J. G. (2006). Effects of a math-enhanced curriculum and 

instructional approach on the mathematics achievement of agricultural power and technology 
students: An experimental study. Journal of Agricultural Education, 47(3), 81-93. doi: 
10.5032/jae.2006.03081 

Parr, B. A., Edwards, M. C., & Leising, J. G. (2009). Selected effects of a curriculum integration 

intervention on the mathematics performance of secondary students enrolled in an agricultural 
power and technology course: An experimental study. Journal of Agricultural Education, 50(1), 
57-69. doi: 10.5032/jae.2009.01057 

Parr, B., & Edwards, M. C. (2004). Inquiry-based instruction in secondary agricultural education: Problem¬ 
solving - An old friend revisited. Journal of Agricultural Education, 45(4), 106-117. doi: 
10.5032/jae.2004.04106 

Provasnik, S., Gonzales, P., & Miller, D. (2009). U.S. performance across international assessments of 
student achievement: Special supplement to the condition of education 2009. NCES 2009-083: 
National Center for Education Statistics. 

Ricketts, J. C., Duncan, D. D., & Peake, J. B. (2006). Science achievement of high school students in 

complete programs of agriscience education. Journal of Agricultural Education, 47(2), 48-55. doi: 
10.5032/jae.2006.02048 

Roberts, T. G., & Ball, A. L. (2009). Secondary agricultural science as content and context for teaching. 
Journal of Agricultural Education, 50(1), 81-91. doi: 10.5032/jae.2009.01081 

Roegge, C. A., & Russell, E. B. (1990). Teaching applied biology in secondary agriculture: Effects on 
student achievement and attitudes. Journal of Agricultural Education, 3i(l), 27-31. doi: 
10.5032/jae. 1990.01027 

Shavelson, R. J. (1996). Statistical reasoning for the behavioral sciences: With additional study guide 
materials. Boston, MA: Pearson Custom Publications. 

Soper, D. (2010). The free statistics calculators website. Retrieved from 
http://www.danielsoper.eom/statcalc/default.aspx#cl6 

Thalheimer, W., & Cook, S. (2002). Plow to calculate effect sizes from published research articles: A 
simplified methodology. Retrieved from http://work-learning.com/effect_sizes.htm 

Thompson, B. (2002). What future quantitative social science research could look like: Confidence intervals 
for effect sizes. Educational Researcher, 31(3), 25-32. doi: 10.3102/0013189X031003025 

Thompson, G. W., & Balschweid, M. M. (2000). Integrating science into agriculture programs: Implications 
for addressing state standards and teacher preparation programs. Journal of Agricultural Education, 
41(2), 73-80. doi: 10.5032/jae.2000.02073 


Journal of Agricultural Education 


26 


Volume 53, Number 2, 2012 


Haynes, Robinson, Edwards, & Key 


Assessing the Effect... 


Trusty, J., Thompson, B., & Petrocelli, J. V. (2004). Practical guide for reporting effect size in quantitative 
research in the journal of counseling & development. Journal of Counseling & Development , §2(1), 
107-110. 

Young, R. B., Edwards, M. C., & Leising, J. G. (2009). Does a math-enhanced curriculum and instructional 
approach diminish students’ attainment of technical skills? A year-long experimental study in 
agricultural power and technology. Journal of Agricultural Education, 50(1), 116-126. doi: 
10.5032/jae.2009.01116 


J. CHRIS HAYNES is an Assistant Professor in the Department of Secondary Education/Agricultural 
Education at the University of Wyoming, McWhinnie Hall 114, Dept. 3374, 1000 E. University Ave., 
Laramie, WY 82071, jhaynes4@uwyo.edu 

J. SHANE ROBINSON is an Associate Professor in the Department of Agricultural Education, 
Communications and Leadership at Oklahoma State University, 457 Ag Hall, Stillwater, OK 74078-6032, 
shane.robinson @ okstate.edu 

M. CRAIG EDWARDS is a Professor in the Department of Agricultural Education, Communications and 
Leadership at Oklahoma State University, 456 Ag Hall, Stillwater, OK 74078-6032, 
craig.edwards@okstate.edu 

JAMES P. KEY is a Professor in the Department of Agricultural Education, Communications and 
Leadership at Oklahoma State University, 545C Ag Hall, Stillwater, OK 74078-6032, 
james.key@okstate.edu 


Journal of Agricultural Education 


27 


Volume 53, Number 2, 2012 


